{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 信息熵Entropy\n",
    "## 信息熵的特点\n",
    "> 熵在信息论中代表随机变量不确定度的度量，可以联想下老板经常说地熵增和熵减\n",
    "\n",
    "+ 熵越大，数据的不确定性越高\n",
    "+ 熵越小，数据的不确定性越低\n",
    "## 信息熵的公式\n",
    "+ k代表分类数\n",
    "+ pi代表第i个分类的概率，范围0~1\n",
    "+ 最前面有负号是因为`log(pi)`一定是小于0的\n",
    "![信息熵的公式](images/信息熵的公式.png)\n",
    "信息熵公式的举例举例如下：\n",
    "+ 左边的数据都是均分地，数据落在每个里面的概率是相等地，给定一个新元素很难估计在哪个里面，所以这个例子很随机，信息熵更大\n",
    "+ 右边的数据第3部分占70%，给定一个新数据我们有很大把握说是在第3部分里，所以这个例子更不随机，信息熵更小\n",
    "![信息熵公式的举例](images/信息熵公式的举例.png)\n",
    "极端的例子，当其中一个概率为1时，所有数据的归属都是确定的，随机不确定度就为0了\n",
    "![信息熵为0的例子](images/信息熵为0的例子.png)\n",
    "\n",
    "## 信息熵公司的推导\n",
    "![信息熵的公式2](images/信息熵的公式2.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def entropy(p):\n",
    "    return -p * np.log(p) - (1-p) * np.log(1-p)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.linspace(0.01, 0.99, 200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3dd3yV9d3/8dfnZCdkQQYhCSSBBAggK2wR3KgV3IKTOqhaa2vXrbW1/dm749ZWe9diLa5Wq6J1oqLgwMEmQBhhJszsQXZCxsn390di7xQDOZCcc53xeT4eeTxyzrnMeV8E3n7P9/pe1yXGGJRSSnk+m9UBlFJK9Q0tdKWU8hJa6Eop5SW00JVSyktooSullJfwt+qNY2JiTEpKilVvr5RSHmnz5s0VxpjY7l6zrNBTUlLIzs626u2VUsojicjhk72mUy5KKeUltNCVUspLaKErpZSXcKjQRWSOiOwVkTwReaCb158QkZzOr30iUt33UZVSSp1KjwdFRcQPWAxcCBQAm0RkmTFm19fbGGPu77L994DxTsiqlFLqFBwZoU8G8owxB4wxLcBSYN4ptl8AvNoX4ZRSSjnOkUJPBI52eVzQ+dw3iMgQIBX47CSvLxKRbBHJLi8vP92sSimlTsGRdejSzXMnu+bufOANY4y9uxeNMUuAJQBZWVl63V7lcsYYqhtbKatrpqzuOFWNrRxvsdPU2vnVYkcEAvxsBPgJ/jYbQQE2+ocGEh0WSP+wQKJDAxkQFojN1t0/DaWs40ihFwDJXR4nAUUn2XY+8N3ehlKqt6oaWthTUkd+eT355fXkldVzsKKBstpmWuztvf75gX42kvuHMGRAGIP7h5IaE0bmoAgyEyIIC7LsfD3l4xz5m7cJSBeRVKCQjtK+4cSNRGQ4EA2s69OESvWgzd7OruJath6pZuuRKnKOVnOosvHfr4cG+jE0th8TBkeTEBVMfHgwcRFBxIUH0z8sgJBAf0IC/AgJ8CPIv2MWsrW9nTa7oc1uON5m51hDC1UNLRxrbOFYQwuF1U0crmjk8LFG1h+opLGl40OpCKTFhDE6MZJxyVFMHxpDRnw/RHQ0r5yvx0I3xrSJyL3ACsAPeN4YkysijwDZxphlnZsuAJYavQWScoHC6ia+3FfOl/vKWZNXQe3xNgDiwoMYPziK+ZMHk5kQwbC4fiREBp92oQbZ/Ph6oB1JAPERwSfd1hhDaW0zuUU17CysZWdRDRsPHuPdnI4PsjH9ApmaNoDpQ2M4d0QsCZEhZ7bTSvVArOrfrKwso9dyUacjr6yO97cXs3xHMftK6wFIiAzmnPRYzk6PYeKQ6DMqb2cpqGpkXX4l6/IrWZNfQWltMwBjk6O4eFQ8F48ayNDYfhanVJ5GRDYbY7K6fU0LXbmzo8caeWdrIR/sKGZPSR0iMDmlPxdmxjMrI5ZhcZ4xnWGMIb+8nhW5pazMLWFbQQ0AGfH9uHpCEldOSCQu/OSfApT6mha68iit9nY+2VXKKxuPsDqvAmMga0g03zorgUvHJBB3iukPT1FU3cTK3BKWbStiy5Fq/GzC7IxYrs1K4rwR8QT661U5VPe00JVHKKk5zovrDvF6dgEV9c0kRAZz/aRkrs1KJjHKe+ed88vreWNzAW9tKaC0tpn4iCBumZbCgsmD6R8WaHU85Wa00JVb219ax5IvD/BOTiH2dsN5I+K5YUoyszLi8POhtd72dsMX+8p4Yc0hvtpfQZC/jasmJHHbjBTS48OtjqfchBa6ckubD1fx18/z+GR3GcEBNq7PSuaOmWkk9w+1Oprl9pXW8cKag7y1pZDmtnYuHTOQ+85PZ8TACKujKYtpoSu3kltUwx9X7uOzPWVEhwZw6/QUbpmWotML3aisb+Yfaw/x/JpD1De3cdmYBO47P53hA3XE7qu00JVbOFBezxOf7Oe9bUVEBPtz1+yhLJyeQmignlnZk+rGFp5bfZAX1hyioaWNy88axE/nDCcpWj/N+BotdGWpmsZWnvhkHy+tP0yQv43bZqRy5zlpRIYEWB3N41Q1tPDMVwd4bvVBDHDH2ancc+4w+unlBnyGFrqyhL3d8Nqmozy2Yg81Ta3cOGUI952fTmx4kNXRPF5RdROPfrSHd3KKiOkXxI8vyuDarGSfOojsq7TQlcttPnyMXy7LZWdhLZNT+/Ory0eROUgP6PW1nKPV/Pr9XWw+XMW45Ch+d9UYRibon7M300JXLlPf3MajH+3hxXWHSYgM5meXjuRbZyV4xNmcnsoYw7s5Rfz6/V3UNLVyx8w0vn9+OiGBflZHU05wqkLXiTfVZ77YV87P3tpBUU0T356Rwo8vGq6XknUBEeGK8YnMyojldx/u5ukv8lm+o5jfXDmamemxVsdTLqTnF6teq2ls5Uevb+PW5zcSHGDjjbum8cvLR2mZu1h0WCCPXjOWV++cir9NuPm5jfzinZ00tXR7vxnlhfRfnOqV9Qcquf+1HMrqmvnuuUP53nnpBAfoR30rTRs6gOXfn8kfVuzl2dUHWZNXwePXj2NccpTV0ZST6QhdnZFWezt/XLmXBc+sJ8jfxlt3T+cnF4/QMncTwQF+/PxbmbxyxxSaWu1c/de1/O8n+2nrg7s1Kfelha5O25HKRq772zqe/CyPayYk8cF9Mxmroz+3NH1YDB/94BwuPyuBJz7Zx/wl6ympOW51LOUkWujqtKzILeGyP39FXlk9Ty4Yz2PXjtW5cjcXGRLAn+aP50/Xj2NXcS2X/fkrVu+vsDqWcgItdOUQe7vh0Y/28J2XNpMWG8aH35/J5WMHWR1LnYYrxiey7N4ZRIcFcvPzG/jzp/tpb9c7RnoTLXTVo2MNLSx8YSNPfZ7PgsnJvPadaXoNEQ81LC6cd787g3ljB/H4x/tY+PdN1DS2Wh1L9REtdHVKOwtruPzJ1Ww4eIz/uXoMv7vqLD3w6eHCgvx54vpx/ObK0azLr+CKp9aQX15vdSzVB7TQ1UmtyC3hmqfXYozhjbumcf2kwVZHUn1ERLhxyhBeuXMqNU2tXLl4DV/tL7c6luolLXT1DcYYlnyZz13/3MyIgRG8e+/ZnJWkq1i80aSU/rz73RkkRIaw8IVNvLjukNWRVC84VOgiMkdE9opInog8cJJtrhORXSKSKyKv9G1M5Sqt9nYeemcnv12+h0tHJ7B00VS9OqKXS+4fypv3TOfc4bE8/G4uv3x3J3Y9WOqRelxvJiJ+wGLgQqAA2CQiy4wxu7pskw48CMwwxlSJSJyzAivnqTveyj0vb+Gr/RXcM3soP75oODa9HKtP6Bfkz99uzuL3H+7mma8OUl7fzOPXjdPjJR7GkQXEk4E8Y8wBABFZCswDdnXZ5k5gsTGmCsAYU9bXQZVzVdY3s/CFTewuruXRq8/iuknJVkdSLuZnEx66LJP4iGD++4PdVNZv5Jlbs4gI1huReApHplwSgaNdHhd0PtdVBpAhImtEZL2IzOnuB4nIIhHJFpHs8nI9AOMuiqqbuPZv69hXWseSWyZqmfu4O2am8b/zx7HlSBXXPb2O0lo9s9RTOFLo3X3mPnGCzR9IB2YDC4BnReQbR9GMMUuMMVnGmKzYWL2spzvIL6/nmr+upby2mZdun8J5I+KtjqTcwLxxiTx36ySOHGvkqqfWcqiiwepIygGOFHoB0HXIlgQUdbPNu8aYVmPMQWAvHQWv3NjOwhqufXodLfZ2Xl00lcmp/a2OpNzIORmxLF00laZWO9cvWUdema5Vd3eOFPomIF1EUkUkEJgPLDthm3eAcwFEJIaOKZgDfRlU9a2co9UseGY9IQF+/Ouu6YxOjLQ6knJDZyVF8eqdU7G3G+YvWc++0jqrI6lT6LHQjTFtwL3ACmA38LoxJldEHhGRuZ2brQAqRWQXsAr4iTGm0lmhVe/kHK3m5uc2EB0ayOt3TSM1JszqSMqNDR8YztJF07AJzF+ynl1FtVZHUieh9xT1MV3L/NVFU0mMCrE6kvIQBysauOGZ9TS12vnn7VP0U51FTnVPUT1T1IdomaveSI0J47VF0wgL9OeGZ9azu1hH6u5GC91HbC/QMle9N3hAKEsXTSU00J+bn9ugF/VyM1roPiCvrI5bn99IZEiAlrnqteT+obx85xQAbnp2A0ePNVqcSH1NC93LFVQ1ctOzG/Gz2fjn7VO0zFWfGBrbjxdvm0JDcxs3PbdBTz5yE1roXqyivpmbn9tIY0sbL90+mRRdzaL6UOagCP5x22Qq6pq56dkNHGtosTqSz9NC91K1x1u55bmNFNc08cK3JzEyIcLqSMoLjR8czXMLO84ove3vm2hqsVsdyadpoXuh46127vhHNvvL6nj6polMHKJngCrnmZo2gD8vGM/2gmq+9+pW2uztVkfyWVroXqa93fDjf21j48Fj/PG6ccwerlcyVs538aiB/GruKD7ZXcovl+Vi1fktvs6Ry+cqD/KHlXt5f3sxD1wygrljB1kdR/mQW6alUFR9nKe/yGdQVAjfPXeY1ZF8jha6F1m68QhPfZ7PgsmD+c45aVbHUT7opxcPp6SmicdW7CUhMpirJiRZHcmnaKF7iS/3lfPQOzs5JyOWX88bhYjeaUi5ns0mPHrNWMrqmvnpG9tJjAphStoAq2P5DJ1D9wJ7Smq55+UtpMf1Y/EN4/H301+rsk6gv42/3jSRwQNCufvlLXrikQvpv3wPd6yhhTv+kU1ooB/PL5xEuN4uTLmByJAAnr0lizZ7O3e+mE19c5vVkXyCFroHa7O3c+8rWyira+aZW7IYpGeBKjeSFtuPxTdOYH9ZPfe/lkN7u658cTYtdA/22+V7WJtfyW+uGM3Y5G/c8U8py81Mj+Xnl43k412l/PHjvVbH8Xp6UNRDvbm5gOfXHGTh9BSuzdKbOiv3tXB6CntL6li8Kp+M+HDmjTvxHvOqr+gI3QNtO1rNg2/vYFraAB66bKTVcZQ6JRHhkXmjmZQSzQNv7tDb2DmRFrqHKa9r5jsvbSa2XxCLb5xAgK5oUR4g0N/G4hsmEBbkz13/3KwHSZ1E28CD2NsN97+WQ1VjC0tumUj/sECrIynlsLiIYJ5cMJ5DFQ381xvb9fIATqCF7kH+8lkeq/Mq+H9zRzFqkN7PUXmeaUMH8JOLR/DBjmJeWHPI6jheRwvdQ6zNq+BPn+7jyvGJXD9JD4Iqz3XXrDQuzIznt8t3s/nwMavjeBUtdA9QVnec+5bmkBYTxn9fMVpP61ceTUT4w7VjSYwO4Z6Xt1BZ32x1JK/hUKGLyBwR2SsieSLyQDevLxSRchHJ6fy6o++j+iZ7u+H7r+ZQ39zKUzdOJCxIV5oqzxcZEsBTN06gqrGVn+h8ep/psdBFxA9YDFwCZAILRCSzm01fM8aM6/x6to9z+qz//XQ/6w5U8ut5oxk+MNzqOEr1mVGDInnwkhF8tqeMF9cdtjqOV3BkhD4ZyDPGHDDGtABLgXnOjaUANh06xl8+28/VE5L05CHllRZOT+Hc4bH8ZvludhfXWh3H4zlS6InA0S6PCzqfO9HVIrJdRN4QkW7bR0QWiUi2iGSXl5efQVzfUXu8lftfyyEpOpT/N2+U1XGUcgoR4bFrxxIRHMB9r27Ve5L2kiOF3t0RuBMnvN4DUowxZwGfAP/o7gcZY5YYY7KMMVmxsbGnl9TH/GpZLsU1x3ni+nH003lz5cVi+gXx+HVj2V9Wz2+W77I6jkdzpNALgK4j7iSgqOsGxphKY8zXh6qfASb2TTzf9P72It7aUsi95w5j4pBoq+Mo5XTnZMSy6Jw0/rn+CCtyS6yO47EcKfRNQLqIpIpIIDAfWNZ1AxFJ6PJwLrC77yL6luKaJh56eyfjkqP43nl6T0blO3580XBGJ0bwwJvbKa/TpYxnosdCN8a0AfcCK+go6teNMbki8oiIzO3c7D4RyRWRbcB9wEJnBfZm7e2GH72+jVZ7O3+6fpzeeUj5lEB/G09cN46GFjs/f2eHLmU8Aw5NzhpjlgPLT3ju4S7fPwg82LfRfM/f1x5ibX4l/3P1GFJiwqyOo5TLpceH86MLM/jdh3t4N6eIK8brpXZPhw4B3cShigYeXbGH80bEcZ0uUVQ+7I6ZaUwYHMUvl+VSWnvc6jgeRQvdDbS3G3765nYC/Gz89soxemq/8ml+to5LAzS32XnwLZ16OR1a6G7gnxsOs/HgMX5xWSYDI4OtjqOU5dJi+/HTizvOIn1jc4HVcTyGFrrFjh5r5Pcf7uGcjFiuzUqyOo5SbmPh9BQmp/bnkfd2UVTdZHUcj6CFbiFjDA+8tR2bCL+7SqdalOrKZhP+cM1Y2toND7+7U6deHKCFbqFXNx5lTV4lD146gsSoEKvjKOV2Bg8I5YcXZvDJ7jI+2qknHPVEC90ipbXH+e3y3UwfOoAbJg+2Oo5SbuvbM1IYNSiCXy7Lpaap1eo4bk0L3SKPvL+LFnu7rmpRqgf+fjZ+f9VZVNQ38+hHe6yO49a00C3w+d4yPthezPfOHaYnECnlgDFJkXx7RiovbzhC9iG9bd3JaKG72PFWOw+/m0tabBiLZqVZHUcpj/HDCzNIjArhwbd20NLWbnUct6SF7mKLV+Vx5Fgj/33FaIL8/ayOo5THCAvy59dXjGJ/WT1/+yLf6jhuSQvdhfLK6nn6i3yuGp/I9KExVsdRyuOcNyKey85K4MlVeRypbLQ6jtvRQncRYww/f2cHIQF+/OyykVbHUcpj/eKyTPxtwiPv680wTqSF7iJvby1k/YFjPHDJSGL6BVkdRymPNTAymPvOT+eT3aWs2ltmdRy3ooXuAvXNbfzuwz2MS45i/iS9kqJSvXXbjFTSYsJ45L1dNLfpfUi/poXuAotX5VFe18yv5o7CZtM150r1VqC/jYcvz+RgRQPPrz5kdRy3oYXuZIcrG3juq4NcPSGJcclRVsdRymvMHh7HhZnxPPnZfkpq9LrpoIXudL/5YDcBfsJ/zRludRSlvM4vLsukrd3w2+V6G2PQQneq1fsrWLmrlO+eN4y4CL3OuVJ9bfCAUO46J41l24rYcKDS6jiW00J3kjZ7O4+8n8vg/qHcNiPV6jhKea27Zw8jMSqER97fRXu7b19iVwvdSV7ZeIR9pfU8dNlIggP0jFClnCUk0I+fXDyc3KJa3t1WaHUcS2mhO0F1YwuPf7yPGcMGcFFmvNVxlPJ6c8cOYnRiBI99tJfjrb67jNGhQheROSKyV0TyROSBU2x3jYgYEcnqu4ie56nP86lpauUX38rUS+Mq5QI2m/CzS0dSVHOcF9YcsjqOZXosdBHxAxYDlwCZwAIRyexmu3DgPmBDX4f0JIXVTfx97SGunpDEiIERVsdRymdMHxrD+SPieGpVHpX1zVbHsYQjI/TJQJ4x5oAxpgVYCszrZrtfA48CPr0g9ImP9wFw/4UZFidRyvc8cMkIGlraePKzPKujWMKRQk8EjnZ5XND53L+JyHgg2Rjz/ql+kIgsEpFsEckuLy8/7bDubk9JLW9uKWDh9BS9R6hSFkiPD2f+5MH8c/1hDlY0WB3H5Rwp9O4mgf+9NkhEbMATwI96+kHGmCXGmCxjTFZsbKzjKT3Eox/tJTzIn3tmD7U6ilI+6wcXpBPob+N/PvS929U5UugFQNcrSiUBRV0ehwOjgc9F5BAwFVjmawdG1x+o5LM9Zdxz7jCiQgOtjqOUz4oLD+auWUP5KLeEzYerrI7jUo4U+iYgXURSRSQQmA8s+/pFY0yNMSbGGJNijEkB1gNzjTHZTknshowx/P7DPQyMCGbh9BSr4yjl824/O5UBYYE8/vFeq6O4VI+FboxpA+4FVgC7gdeNMbki8oiIzHV2QE/w0c4Sco5W88MLM/QkIqXcQFiQP3fPHsqavErW5ldYHcdlxBhrTpXNysoy2dmeP4i3txsueuILbCJ8+P2Z+PvpuVpKuYPjrXZmPbaK5OhQ/nXXNK85J0RENhtjup3S1vbppfe2FZFf3sD9F2ZomSvlRoID/Lj3vHSyD1fxxT7vW1XXHW2gXmizt/PnT/czYmA4c0YNtDqOUuoE12clkxQdwh9X7sOq2QhX0kLvhWXbijhQ0cAPLkjXOxEp5YYC/W18//x0dhTWsHJXqdVxnE4L/Qx9PTofmRDBRZk6OlfKXV05PpG0mDAeX7nP6y+vq4V+ht7JKeJQZaOOzpVyc/5+Nn5wYQZ7S+t4f0ex1XGcSgv9DLTZ23nys/2MGhShl8dVygN8a0wCw+PD+dMn+7B78ShdC/0MvLW1kMOVjfzgggyvWQqllDez2YR7zxvGgfIGPtpZYnUcp9FCP02tnaPzMYmRXDAyzuo4SikHXTomgbSYMP6yKs9rV7xooZ+mt7cWcvRYE/dfmK6jc6U8iJ9NuHv2UHYX17Jqb5nVcZxCC/00tLcbnv4in8yECM4drqNzpTzNFeMTSYwK4S+feecoXQv9NKzcVcqB8gbunj1UR+dKeaAAPxt3zUpjy5Fq1h2otDpOn9NCd5Axhr9+kc+QAaFcMlrXnSvlqa7NSiY2PIjFq7zvrkZa6A5ad6CSbUerWXROml6zRSkPFhzgx50zU1mTV8nWI951vXRtJgf99fN8YvoFcfWEJKujKKV66cYpQ4gKDfC6UboWugN2FNTw1f4Kbj87Va93rpQXCAvy57YZqXyyu4xdRbVWx+kzWugOePqLfMKD/Llx6mCroyil+sit01IIDfTj2dUHrI7SZ7TQe3CwooHlO4u5edoQIoIDrI6jlOojkaEBXJeVzHvbiiitPW51nD6hhd6DJV/mE+Bn49szUq2OopTqY7fNSMXebvjH2kNWR+kTWuinUF7XzJubC7l2YhKx4UFWx1FK9bHBA0K5eNRAXt5whMaWNqvj9JoW+im8suEILfZ2bjtbR+dKeas7ZqZS09TKG5sLrI7Sa1roJ9HcZuel9YeZPTyWobH9rI6jlHKSCYOjGZccxfOrD3r8pXW10E/ig+3FVNQ3c5vOnSvl1USEO2amcqiykU93e/Zt6hwqdBGZIyJ7RSRPRB7o5vW7RGSHiOSIyGoRyez7qK5jjOH5NQcZFtePmekxVsdRSjnZnFEDSYwK4dmvDlodpVd6LHQR8QMWA5cAmcCCbgr7FWPMGGPMOOBR4PE+T+pC2Yer2FlYy7dnpOhFuJTyAf5+Nr49I4WNh46x7Wi11XHOmCMj9MlAnjHmgDGmBVgKzOu6gTGm66lWYYBHT0Q9v/ogkSEBXDVeT/NXyldcPymZ8CB/nl3tuaN0Rwo9ETja5XFB53P/QUS+KyL5dIzQ7+vuB4nIIhHJFpHs8vLyM8nrdAVVjazILWHB5MGEBOpp/kr5ivDgAK6flMzyHcWU1HjmiUaOFHp3cw7fGIEbYxYbY4YC/wX8vLsfZIxZYozJMsZkxcbGnl5SF3lx3WFEhFumDbE6ilLKxW6eNoR2Y3hl4xGro5wRRwq9AEju8jgJKDrF9kuBK3oTyioNzW0s3XiEOaMHMigqxOo4SikXGzIgjNkZsby68Qgtbe1WxzltjhT6JiBdRFJFJBCYDyzruoGIpHd5eBmwv+8ius5bWwupPd7GbTNSrI6ilLLILdNSKK9rZkVuidVRTluPhW6MaQPuBVYAu4HXjTG5IvKIiMzt3OxeEckVkRzgh8CtTkvsJMYYXl5/mNGJEUwYHG11HKWURWZlxDK4fygvrTtsdZTT5u/IRsaY5cDyE557uMv33+/jXC63+XAVe0rq+P1VY3SpolI+zGYTbpo6mN8u38OeklpGDIywOpLD9EzRTi9vOEJ4kD+Xjx1kdRSllMWuy0omyN/Gix42StdCB441tPDBjmKunJBIWJBDH1qUUl4sKjSQeeMG8faWQuqOt1odx2Fa6MCbmwtoaWvnhil6RyKlVIcbpgyhqdXOsm2nWtTnXny+0NvbO9acZg2J9qi5MqWUc41NimRkQgSvetCadJ8v9LX5lRysaOCmqXoikVLq/4gICyYns7Owlh0FNVbHcYjPF/qrG48QFRrAnNEDrY6ilHIz88YlEhxg49VNnjFK9+lCP9bQwspdJVw5PpHgAL1ui1LqP0WGBHDpmASW5RTR0Oz+t6jz6UJ/Z2shrXbD9ZOSe95YKeWTbpg8mPrmNt7f7v4HR3220I0xvJ59lLOSIvVgqFLqpCYOiWZYXD9e3Xi0540t5rOFvqOwhj0ldVyXpaNzpdTJiQjzJyWTc7SafaV1Vsc5JZ8t9Nc2HSXI38bccXpmqFLq1K4Yn4i/TXhzc4HVUU7JJwu9qcXOspwiLh2TQERwgNVxlFJuLqZfELOHx/HW1kLa7O57WV2fLPSPcoupa27T6RallMOumZhEeV0zX+2vsDrKSflkob+xuYDB/UOZktrf6ihKKQ9x3og4okMDeGOL+067+FyhF9c0sTa/kivHJ2Kz6WVylVKOCfS3MW9cIh/nllLT6J4X7PK5Ql+WU4QxHQc5lFLqdFwzMYkWezvvuemadJ8r9Le3FjJ+cBSpMWFWR1FKeZhRgyIYMTCcN9x0tYtPFfru4lr2lNRxlY7OlVJnQES4ekISOUeryS+vtzrON/hUob+9tRB/m3DZWbr2XCl1ZuaOG4RIx/Stu/GZQre3G97NKWT28Dj6hwVaHUcp5aHiI4KZmjqA97YVYYyxOs5/8JlCX5dfSWltM1dN0OkWpVTvzBs3iAMVDewsrLU6yn/wmUJ/e2sh4cH+nDcizuooSikPd8noBAL8hHdzCq2O8h8cKnQRmSMie0UkT0Qe6Ob1H4rILhHZLiKfiohb3f7neKudlbklzBk1UK97rpTqtcjQAGZlxPHe9iLs7e4z7dJjoYuIH7AYuATIBBaISOYJm20FsowxZwFvAI/2ddDe+Gp/BXXNbXxrrB4MVUr1jXnjBlFa28zGg8esjvJvjozQJwN5xpgDxpgWYCkwr+sGxphVxpjGzofrgaS+jdk7728vIjo0gOlDB1gdRSnlJS4YGU9ooB/LtrnPtIsjhZ4IdL2ye0HncydzO/Bhdy+IyCIRyRaR7PLycsdT9sLxVjuf7CplzuiBBPj5zCEDpZSThQT6cVFmPMt3lNDS5h5XYHSk4bq74Em3k0YichOQBTzW3evGmCXGmCxjTFZsbKzjKXth1Z4yGpa1CogAAAqSSURBVFrsfEvXniul+tjccYOoaWpldZ5rBqg9caTQC4Cu15lNAr6xol5ELgAeAuYaY5r7Jl7vvb+jmJh+gXplRaVUn5sxLIbwIH8+3FFidRTAsULfBKSLSKqIBALzgWVdNxCR8cDf6Cjzsr6PeWYaW9r4bHcZc0YPxF+nW5RSfSzI348LMuP5eHcprW5w44seW84Y0wbcC6wAdgOvG2NyReQREZnbudljQD/gXyKSIyLLTvLjXOrT3WU0tep0i1LKeeaMHkh1YyvrD1RaHQV/RzYyxiwHlp/w3MNdvr+gj3P1iQ+2FxMbHsSkFJ1uUUo5x6yMWEID/fhwZwkz011zbPBkvHYeoqnFzuf7ypgzaiB+eiMLpZSTBAf4cd6IOFbmllh+kpHXFvrqvAqOt7Zz8aiBVkdRSnm5S0YnUFHfwqZD1p5k5LWFviK3hPBgf6ak6XSLUsq5Zg+PJTjAxkc7rV3t4pWF3mZv59PdpZw/Ik5PJlJKOV1YkD+zMmL5cGcx7RZOu3hl22UfrqKqsZWLdLpFKeUil4xOoLS2ma1Hqy3L4JWFvjK3lEB/G7MyrD3irJTyHeeOiMPPJny6u9SyDF5X6MYYVu4qYeawGMKCHFqVqZRSvRYZEsCklGg+3W3duZVeV+i7imspqGriolHxVkdRSvmYC0bGs7e0jqPHGnve2Am8rtBX5pYiAueP1EJXSrnW171j1bSL1xX6x7tKmTg4mph+QVZHUUr5mNSYMNJiw/h0jzXTLl5V6KW1x9lVXKujc6WUZS4YGc/6A5XUHW91+Xt7VaF/sbfjmsSzh+vqFqWUNc4fEUer3fDV/gqXv7dXFfqqvWUMjAhmxMBwq6MopXzUxCHRRIYE8IkF8+heU+it9nZW769g9vBYRPRiXEopa/j72Zg9PJbP95a7/GJdXlPomw9XUdfcxuzhcVZHUUr5uPNHxnOsoYWtR6pc+r5eU+if7y3H3ybMGDbA6ihKKR83Kz0Wm8CXLp5H96JCLyMrJZrw4ACroyilfFxkaABjkqL4ar9rbx7tFYVeXNPEnpI6ztXpFqWUmzgnPYZtR6upaXLd8kWvKPTP/71cUQtdKeUeZqbH0m5gXb7rpl28pNDLGBQZTEZ8P6ujKKUUAOMHRxEW6OfS9egeX+ht9nbW5lUyS5crKqXcSICfjWlDB2ihn46dRbXUNbcxfWiM1VGUUuo/zEyP5cixRg5XNrjk/RwqdBGZIyJ7RSRPRB7o5vVzRGSLiLSJyDV9H/Pk1uVXAjA1TZcrKqXcy9npHQNNV43Seyx0EfEDFgOXAJnAAhHJPGGzI8BC4JW+DtiTtfkVZMT3IzZcr66olHIvaTFhJEaFuGz5oiMj9MlAnjHmgDGmBVgKzOu6gTHmkDFmO9DuhIwn1dLWTvahKqbp6Fwp5YZEhJnpMazNq6TN7vx6dKTQE4GjXR4XdD532kRkkYhki0h2eXnv/4+1raCaplY703T+XCnlps5Oj6GuuY1tBc6/ebQjhd7d0pEzuuKMMWaJMSbLGJMVG9v7S9yuy69EBKam9e/1z1JKKWf4esHG18f7nMmRQi8Akrs8TgKKnBPn9KzNryAzIYKo0ECroyilVLf6hwUyYmA4Gw4ec/p7OVLom4B0EUkVkUBgPrDMubF6drzVzpYj1Tp/rpRye1PTBpB9qIpWJ8+j91joxpg24F5gBbAbeN0Ykysij4jIXAARmSQiBcC1wN9EJNeZoQG2HK6ipa2d6Xp1RaWUm5uS2p+mVjvbC2qc+j7+jmxkjFkOLD/huYe7fL+JjqkYl1l3oBI/mzApRefPlVLubXJqR09tOFjJxCHRTnsfjz1TdG1+JWMSI/VyuUoptzegXxAZ8f1Yf8C58+geWejHW+1sL6jWs0OVUh5jSuoANh865tTb0nlkoW87Wk2r3ZDlxI8uSinVl7JSomlosbOnpNZp7+GRhb658z59zpyLUkqpvjRhcEdfbTnsvPuMemahH6piaGwY0WG6/lwp5RmSokOIjwgiWwv9/xhj2HykSkfnSimPIiJMHBLNZi30/5Nf3kB1YytZQ3S5olLKs0wc0p+CqiZKa4875ed7XKF/Pf80QUfoSikP8/XMgrNG6R5X6FGhAVyYGc/Q2DCroyil1GkZNSiC80bEERbk0Dmdp02Mcd6ayFPJysoy2dnZlry3Ukp5KhHZbIzJ6u41jxuhK6WU6p4WulJKeQktdKWU8hJa6Eop5SW00JVSyktooSullJfQQldKKS+hha6UUl7CshOLRKQcOHwa/0kMUOGkOO5M99v3+Oq+6347ZogxJra7Fywr9NMlItknOzvKm+l++x5f3Xfd797TKRellPISWuhKKeUlPKnQl1gdwCK6377HV/dd97uXPGYOXSml1Kl50ghdKaXUKWihK6WUl3C7QheROSKyV0TyROSBbl4PEpHXOl/fICIprk/Z9xzY7x+KyC4R2S4in4rIECty9rWe9rvLdteIiBERr1jW5sh+i8h1nb/zXBF5xdUZncGBv+eDRWSViGzt/Lt+qRU5+5qIPC8iZSKy8ySvi4j8ufPPZbuITDijNzLGuM0X4AfkA2lAILANyDxhm3uApzu/nw+8ZnVuF+33uUBo5/d3+8p+d24XDnwJrAeyrM7tot93OrAViO58HGd1bhft9xLg7s7vM4FDVufuo30/B5gA7DzJ65cCHwICTAU2nMn7uNsIfTKQZ4w5YIxpAZYC807YZh7wj87v3wDOFxFxYUZn6HG/jTGrjDGNnQ/XA0kuzugMjvy+AX4NPAo451bprufIft8JLDbGVAEYY8pcnNEZHNlvA0R0fh8JFLkwn9MYY74Ejp1ik3nAi6bDeiBKRBJO933crdATgaNdHhd0PtftNsaYNqAGGOCSdM7jyH53dTsd/zf3dD3ut4iMB5KNMe+7MpiTOfL7zgAyRGSNiKwXkTkuS+c8juz3r4CbRKQAWA58zzXRLHe6HdAt59x6+sx1N9I+cV2lI9t4Gof3SURuArKAWU5N5Bqn3G8RsQFPAAtdFchFHPl9+9Mx7TKbjk9jX4nIaGNMtZOzOZMj+70A+Lsx5o8iMg14qXO/250fz1J90mvuNkIvAJK7PE7imx+5/r2NiPjT8bHsVB9lPIEj+42IXAA8BMw1xjS7KJsz9bTf4cBo4HMROUTH3OIyLzgw6ujf83eNMa3GmIPAXjoK3pM5st+3A68DGGPWAcF0XLzK2znUAT1xt0LfBKSLSKqIBNJx0HPZCdssA27t/P4a4DPTeVTBg/W4351TD3+jo8y9YT4VethvY0yNMSbGGJNijEmh49jBXGNMtjVx+4wjf8/foeNAOCISQ8cUzAGXpux7juz3EeB8ABEZSUehl7s0pTWWAbd0rnaZCtQYY4pP+6dYffT3JEd799FxNPyhzuceoeMfMnT8gv8F5AEbgTSrM7tovz8BSoGczq9lVmd2xX6fsO3neMEqFwd/3wI8DuwCdgDzrc7sov3OBNbQsQImB7jI6sx9tN+vAsVAKx2j8duBu4C7uvy+F3f+uew407/neuq/Ukp5CXebclFKKXWGtNCVUspLaKErpZSX0EJXSikvoYWulFJeQgtdKaW8hBa6Ukp5if8PPQisaW42Q4gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x, entropy(x))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 信息熵的目标\n",
    "![信息熵的目标](images/信息熵的目标.png)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
